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The  time  has  arrived  for 
accurately  predicting  programmatic 
cost  and  schedule  risk  on  large 
weapon  system  projects.  Management 
science  and  operations  research 
techniques  coupled  to  power  of 
today's  computer  provide  timely 
decision  information  for 
sophisticated  budgetary  and 
scheduling  strategies.  However,  the 
time  for  using  this  information  in  a 
systematic  fashion  has  not  arrived . 

"JIMS  <i  .if  v.u».  1 t ttt  jikofclt*..  a ^ 

Some  years  ago,  I was  working  on 
the  cost/schedule  budgets  and  risks 
for  a major  weapon  system 
development.  I was  suddenly  summoned 
by  the  newly  appointed  Project 
Manager.  Upon  arrival  he  promptly 
asked  "Mr.  Cockerham,  what  is  the 
probability  that  I will  bring  this 
program  in  on  schedule?"  With 
surgical  precision,  I replied  “Zero, 
Sir."  He  asked  “What  is  the 
probability  that  I will  bring  this 
project  in  within  costs?"  Again,  my 
reply,  "Zero,  Sir."  He  raised  hie 
voice  and  asked  "Well  then,  what  is 
the  good  news?"  I answered,  "That  was 
the  good  newsl"  In  looking  back  at 
the  exchange,  the  Project  Manager  was 
just  trying  to  learn  something  about 
his  program  risks  and  I was  of  the 
mind  set  to  simply  answer  the 
question  and  no  more.  This  was  a 
rather  meaningless  exchange  in  that 
probability  alone  did  not  indicate 
the  risks  nor  help  the  Project 
Manager  better  understand  his 
program . 

To  illustrate  the  point  a risk 
profile  for  the  schedule  of  project 
is  shown  on  Figure  1. 


FIGURE  1 


SCHEDULE  ( MONTHS ) 

This  does  not  mean  that  the; 
Planned  schedule  should  be 
rescheduled  to  forty  eight  (48) 
months.  On  the  contrary,  the  thirty 
eight  (38)  month  Planned  Schedule  is 
used  to  drive  the  program  to  the 
earliest  possible  conclusion.  This 
strategy,  in  fact,  serves  a useful 
purpose  but  unfortunately,  total 
program  dollars  are  often  tied  to  a 
zero  probability  schedule  which  of 
course  yields  a zero  probability 
budget.  Such  is  just  one  of  the 
numerous  examples  of  the  difficulties 
in  communicating  with  probabilistic 
information.  Even  when  these 
difficulties  are  overcome  at  the 
program  level,  the  Project  Manager 
is,  at  best,  reluctant  to  communicate 
this  information  to  higher  levels . 
This  can  be  easily  understood  when 
one  looks  at  some  of  the  type 
information  yielded  from  a 
probabilistic  analynis.  The 
following  examples  are  true 
statements  concerning  some  well 
planned  programs . - - 

"Probability  of  meeting  the 

schedule  is  zero." 


An  Appropriate  Statement  of  Risk: 

"There  is  a ( .5)  probability  that 
the  program  will  take  up  to  ten 
(10)  months  additional  time  to 
complete  than  planned." 


"Probability  of  meeting  the  planned 
cost  is  zero." 

"Incremental  funding  of  some  long 
lead  items  for  production  should  be 
initiated  prior  to  the  development 
program. " 
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"$100M  of  RDTfcE  funding  is  needed 
for  the  planned  program  two  years 
after  the  IOC." 

"There  are  negative  cost  risk  in 
the  middle  years  of  the  Engineering 
Development . " 
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"The  cost  riek  ie  greater  than  the 
planned  project  budget." 

Although  there  are  inherent 
difficulties  in  communicating  risk 
information  the  primary  difficulties 
in  DoD  are  presented  by  a system  of 
compartmental  decision  making  that  is 
steeped  in  tradition,  power 
structures  and  resistance  to  change. 
Although  frustrating  at  times,  it  is 
recognized  that  the  aspects  of  our 
system  that  give  reasons  to  problems, 
also  give  reasons  to  much  success. 
Nevertheless,  there  are  substantial 
problems  in  implementing  risk 
information  to  the  DoD  decision 
making  process • 

The  problem  begins  at  the 
Congressional  level  in  that  there  are 
no  requirements  for  uncertainty  or 
risk  information  to  support  the 
Congressional  responsibility  of 
deciding  which  programs  are  funded 
and  how  much.  Moreover,  the 
Congressional  decisions  are 
intertwined  with  the  political 
process  which  customarily  yields 
compromised  results.  Such  decisions 
are  largely  based  upon  qualitative 
assessments  and  political  values. 
There  are  no  management  science  or 
operation  research  methods  to 
describe  the  Congressional  process. 
However,  it  is  my  contention  that  it 
is  better  to  know  the  program's 
planned  coot  and  risks  than  to  know 
only  the  program's  planned  cost. 

At  DoD  there  has  been  no 
shortage  of  words  written  and  words 
spoken  to  the  need  to  analyze,  plan 
and  budget  for  program  uncertainties. 
Lacking  in  these  words  are  firm 
instructions,  guidance,  requirements 
and  the  propensity  to  use  the 
information. 


The  Defense  Acquisition 
Initiatives  have  stimulated  some 
thinking  and  action  by  the  Services. 
However,  the  action  has  been  tenative 
and  lacking  in  application.  This  is 
understandable  in  that  there  is  no 
coordinated  push  from  DoD,  nor  has 
Congress  expressed  any  approval, 


disapproval  or  even  knowledge  of  the 
efforts  to  budget  and  plan  for  risk. 
Congress  is  hardly  to  blame  since 
there  has  been  no  DoD  spokesman  on 
the  subject  and  service  projects  mask 
the  risk  cost  in  their  budgetary 
submissions  to  Congress. 

The  following  discussion 
addresses  the  Lessons  Learned 
concerning  the  Implementation  of  the 
TRACE  Concept  for  the  Navy.  The 
lessons  learned  are  based  on 
experience  from  the  Array's  TRACE 
program  and  the  Navy's  experience 
since  September  30,  1981.  The 

information  was  generated  by  this 
author  under  a contractual  effort  for 
the  Pilot  Application  of  the  TRACE 
Concept  for  the  Navy,  February,  1983. 
The  lessons  are  generally  applicable 
to  DoD  interest  and  are  categorized 
by  the  following  areas i 

Navy  TRACE  Implementation  Lessons 
Learned  " 


* Fiscal  Management 

* Training/Education 

* Manpower 

* Methodology 

* Application 

* Project  Management 

* Resources 

The  nature  of  the  lesson  is 
described  as  an  observation  with 
support  rationale  and  followed  by 
recommendations.  The  subject  areas 
are  addressed  independently  but  are 
in  fact  interrelated.  Therefore,  the 
acceptancs/ rejection  of  the 
observations  and  recommendations 
should  alpo  be  viewed  collectively. 

Fiscal  Management 

Observation i 


The  Navy  hat  not  developed  the 
fiscal  management  methods  to 
systematically  incorporate  the 
elements  of  the  TRACE  concept.  There 
is  confusion  and  doubt  on  behalf  of 
Project  Managers  on  now  to  prepare 
budgetary  submissions  with  the  Riek 
Cost  Estimate  (RCE)  and  what  adverse 
effect  may  result  when  compared  to 
previous  budgetary  submissions. 

Support ; 

The  Army  developed  a management 
system  in  conjunction  with  the 
methodology  of  the  TRACE  concept. 
The  management  system  described  the 
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who,  what,  whan  and  how  the  Array 
KDTfcE  monies  would  be  managed. 
Organizational  infighting  and 
confusion  were  completely  avoided. 
Though  the  Army  system  has  some 
shortcomings,  the  TRACE  concept  has 
survived  largely  due  to  a 
comprehensive  management  system  from 
the  onset.  The  Navy's  approach  has 
been  overly  cautious  to  not  make 
changes  until  there  is  certainty  that 
the  TRACE  deferral  monies  will  not  be 
rejected  by  Congress.  The  Army  has 
already  provided  the  lesson  that 
Congress  will  not  remove  the  TRACE 
deferral  monies  when  properly 
presented.  However,  the  Navy  has  not 
determined  the  method  of  presentation 
nor  the  means  for  managing  the  money 
thereafter.  Navy  policy  on  these 
matters  should  not  be  mutative  and 
chance  failure  at  each  level  of 
budgetary  review.  Instead  the  lesson 
learned  by  the  Army  should  be  heeded 
and  Navy  policy  established 
accordingly  to  Navy  needs. 

Recommendation i 

Immediate  action  should  be  taken 
to  establish  the  Navy's  management 
system  of  RDTfcE  and  production  TRACE 
deferral  funde.  This  effort  should 
include  how  the  funds  are 
established,  updated,  processed, 
authorised,  expended  and  tracked. 


twenty  (20)  or  Igbs  for 
approximately  one  hour.  Cost 
and  system  analysts  were  given 
two  (2)  day  courses.  All 
training  was  provided  by  the 
same  training  team  to  insure 
consistent  and  exacting 
information. 

2.  As  TRACE  was  initially 
successful  on  a short  term  basis 
due  to  the  educational  program, 
TRACE  was  equally  unsuccessful 
on  a long  term  basis  due  to  a 
major  extent,  to  the  lack  of 
follow-on  training. 

Recommendation i 

The  Navy  should  initiate  an 
intensive  training  and  educational 
program  to  introduce  the  science  and 
management  methods  associated  with 
the  TRACE  concept.  This  short  term 
training  should  be  coordinated  with  a 
Navy  handbook  and  be  similar  in  scope 
to  the  Army's  initial  training 
program.  The  training  should  be 
accomplished  in  no  more  than  six 
months  at  a cost  of  less  than  $100K. 
Subsequent  to  the  initial  training,  a 
plan  should  be  developed  for  a long 
term  in-house  training  capability. 
This  could  be  incorporated  into  the 
mission  of  Navy  or  DoD  schools. 


last. 
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Training/Education 


Observation! 


Training  and  education  at  all 
levels  in  the  Navy  is  currently 
needed . 


The  lessons  learned  by  the  Army 


TRACE  was  initially  successful 
due  to  a comprehensive 
educational  and  training 
program.  Congressmen,  Senators, 
professional  staffers,  and  top 
management  at  DoD,  DA  and  DARCOM 
were  given  15-20  minute 
individual  presentations  on  the 
concept.  Every  Commanding 
General,  Deputy  Commander, 
Project  Manager  and  Deputy 
Project  Manager  was  individually 
given  a 20-30  minute  briefing. 
Management  staffers  at  all 
levels  were  briefed  in  groups  of 


Manpower 

Observation i 

Currently,  no  significant 
manpower  capability  has  been 
manifested  within  any  service  to 
apply  the  scientific  methodology 
supporting  the  TRACE  concept . 


From  Army,  Navy  and  major 
contractor  experience  there  is  no 
organisation  or  job  function  or  job 
code  that  can  readily  be  used  to 
perform  TRACE  type  analyses.  The 
problem  is  independent  o£  any  lack  of 
training  or  education  on  the  subject. 
The  problem  can  be  visualized  in  that 
virtually  all  organizations  are 
divided  functionally  in  elements, 
(i.e.,  cost,  schedule,  program  plans, 
program  management,  test,  logistics, 
quality  assurance,  procurement, 
personnel,  etc.).  However,  a TRACE 
analysis  requires  that  detailed 
analysis  be  performed  across  all 
elements  as  relates  to  cost,  schedule 


and  uncertainties.  Furthermore,  all 
elements  are  modeled  and  analyzed 
together.  This  is  all  possible  due 
to  the  advancements  in  computer 
hardwaro/ software  technology. 
However,  traditional  organizational 
elements  are  not  structured  to  take 
advantage  of  computer  technology 
promoting  integrated  analysis  and 
decision  making. 

Recommendation i 

On  a trial  basis  with  a lead 
command,  the  Navy  should  detail  a 
group  of  four  to  six  individuals  to 
do  acquisition  planning.  This  group 
of  acquisition  planners  would  work  on 
multiple  projects  and  cut  across 
functional  boundaries.  Within  six  to 
twelve  months  a credible  and  useful 
in-house  capability  for  acquisition 
planning  and  TRACE  analyses  could  be 
established. 


Methodology 

Observation » 

Methodologies  in  support  of 
probabilistic  analysis  for  the  TRACE 
concept  have  not  bean  established  by 
the  Navy. 


For  the  analysis  and 
distribution  of  TRACE  deferral  RDTfcE 
monies  the  Navy  has  used  an 
interactive  network  analyzer  known  as 
RISNET  and  methods  of  risk 
enumeration.  These  have  been 
accomplished  on  a contractual  basis 
and  the  Navy  has  not  taken  steps  to 
endorse  nor  establish  an  in-house 
capability.  Having  the  methodology 
in-house  1b  the  moot  essential  part 
of  establishing  a capability.  The 
computer  hardwar e/sof tware  and 
operator's  instructions  must  be  made 
available  and  accessible  for  any 
significant  utilization. 

For  TRACE  production,  there  has 
been  no  effective  methodology 
developed  by  any  Service.  The  areas 
of  production  cost,  cost  overruns  and 
production  risks  are  matters  of  great 
national  concern,  regularly  voiced 
through  the  Congress  and  the  news 
media.  However,  there  is  no 
concerted  effort  to  develop  the 
methodology  to  accurately  predict 
production  costs  and  cost  risk. 


In  summary,  the  RDT6E 
methodology  exists  and  is  available 
to  the  Navy.  The  production 
methodology  does  not  exist. 

Recommendations i 

1.  The  Navy  acquire  the 
ha rdvare/ software  for  RDT&E 
TRACE  analysis  and  make  it 
available  to  all  commands. 

2.  The  Navy  initiate  an  applied 
research  program  on  production 
cost  risk  analysis  for  a ship, 
aircraft,  and  missile  system. 


Application 
Observation i 

The  planned  applications  for 
Navy  projects  are  insufficient  to 
support  the  implementation  of  the 
TRACE  concept. 


The  Carlucci  initiative  requires 
the  services  to  implement  TRACE  or  a 
TRACE  type  system.  The  explicit 
implication  is  that  the  TRACE  concept 
is  to  be  applied  to  all  projects  at 
all  commands.  The  Navy  has  initially 
had  good  experience  in  applying  the 
methodology,  but  only  at  one  command 
and  on  one  project.  Application  to 
programs  must  be  significantly 
increased  if  the  implementation  of 
the  TRACE  concept  is  to  be  a serious 
consideration. 

Recommendations: 

1.  Building  on  the  NAVA1R 
experience,  the  S-3B  application 
should  be  continued  in  order  to 
demonstrate  the  maintenance  of 
the  technology  and  usefulness  on 
a continuing  basis.  In 
addition,  two  new  applications 
should  be  initiated.  The 
projects  should  be  selected 
based  on  a need  for  detailed 
planning  and  costing.  If 
possible,  the  projects  should  be 
of  high  complexity  and  early  in 
the  conceptual  or  development 
phase . 

2.  In  conjunction  with  lessons 
learned  in  Training/Education, 
Methodology,  Manpower  and 
Application,  the  technical 
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the  leading  edge  of  the  computer 
aided  decision  sciences  and  should  be 
pursued  aggressively. 

Recommendations : 

1.  DoD,  through  a lead  Service, 
should  commit  a minimum  of  five 
million  dollars  in  FY84/85  for 
the  specific  advancement  of 
TRACE  methodology  for  RDTfcE  and 
production?  procurement  of 
computer  hardware  and  software? 
and  education  at  all  levels. 

2.  DoD,  through  a lead  Service 
Command,  should  initiate  a Btudy 
to  define  a physical  facility  of 
computers,  visual  screens, 
graphic  terminals,  plotters, 
communication  equipment  and 
software  that  would  provide 
state  of  the  art  planning, 
costing  and  control  of  programs. 
The  system  definition  should 
address  the  schedule  and 
resource  requirements  for  the 
facility,  security,  computer 
hardware,  data  base  and 
operational  software, 
documentation,  training, 
implementation  and  cost  for 
duplicate  facilities. 

Many  problems  face  the 
practical  implementation  of  risk 
information  into  the  decision 
process,  However,  the  last  year 
has  produced  greater  progress 
than  all  previous  years. 
Request  for  proposals  are 
requiring  risk  information  and 
in  at  least  one  case,  cost 
realism  was  evaluated  equal  to 
the  total  cost.  DoD  top 
management,  on  several 
occasions,  has  mentioned  the 
TRACE  concept  to  Congress  in 
testimony  regarding  the 
improvements  to  the  acquisition 
process.  Prime  contractors  are 
using  risk  methodology  to 
enhance  their  proposals  and  risk 
management  methods  to  better 
control  their  projects.  At 
least  one  of  the  services  is 
actively  reviewing  selected 
programs  explicitly  for  cost 
risk.  Most  significantly,  the 
reviews  are  conducted  by  the 
Office  of  the  Under  Secretary. 

Heretofore,  the  exclusion 
of  formal  risk  information  in 
the  decision  process  has  been 


the  greatest  shortfall  in  DoD 
and  congressional  planning. 
Changes  have  begun  and  are 
inevitable  because  the  knowledge 
of  risk  has  been  proven  not  only 
useful  but  absolutely  necessary. 
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